Automatic Detection of Public Development Projects in Large Open Source Ecosystems: An Exploratory Study on GitHub

Hosting over 10 million of software projects, GitHub is one of the most important data sources to study behavior of developers and software projects. However, with the increase of the size of open source datasets, the potential threats to mining these datasets have also grown. As the dataset grows, it becomes gradually unrealistic for human to confirm quality of all samples. Some studies have investigated this problem and provided solutions to avoid threats in sample selection, but some of these solutions (e.g., finding development projects) require human intervention. When the amount of data to be processed increases, these semi-automatic solutions become less useful since the effort in need for human intervention is far beyond affordable. To solve this problem, we investigated the GHTorrent dataset and proposed a method to detect public development projects. The results show that our method can effectively improve the sample selection process in two ways: (1) We provide a simple model to automatically select samples (with 0.827 precision and 0.947 recall); (2) We also offer a complex model to help researchers carefully screen samples (with 63.2% less effort than manually confirming all samples, and can achieve 0.926 precision and 0.959 recall).Comment: Accepted by the SEKE2018 Conferenc

Exploring Multi-Programming-Language Commits and Their Impacts on Software Quality: An Empirical Study on Apache Projects

Context: Modern software systems (e.g., Apache Spark) are usually written in multiple programming languages (PLs). There is little understanding on the phenomenon of multi-programming-language commits (MPLCs), which involve modified source files written in multiple PLs. Objective: This work aims to explore MPLCs and their impacts on development difficulty and software quality. Methods: We performed an empirical study on eighteen non-trivial Apache projects with 197,566 commits. Results: (1) the most commonly used PL combination consists of all the four PLs, i.e., C/C++, Java, JavaScript, and Python; (2) 9% of the commits from all the projects are MPLCs, and the proportion of MPLCs in 83% of the projects goes to a relatively stable level; (3) more than 90% of the MPLCs from all the projects involve source files in two PLs; (4) the change complexity of MPLCs is significantly higher than that of non-MPLCs; (5) issues fixed in MPLCs take significantly longer to be resolved than issues fixed in non-MPLCs in 89% of the projects; (6) MPLCs do not show significant effects on issue reopen; (7) source files undergoing MPLCs tend to be more bug-prone; and (8) MPLCs introduce more bugs than non-MPLCs. Conclusions: MPLCs are related to increased development difficulty and decreased software quality.Comment: Preprint accepted for publication in Journal of Systems and Software, 2022. arXiv admin note: substantial text overlap with arXiv:2103.1169

Mining Architectural Information: A Systematic Mapping Study

Context: Mining Software Repositories (MSR) has become an essential activity in software development. Mining architectural information to support architecting activities, such as architecture understanding and recovery, has received a significant attention in recent years. However, there is an absence of a comprehensive understanding of the state of research on mining architectural information. Objective: This work aims to identify, analyze, and synthesize the literature on mining architectural information in software repositories in terms of architectural information and sources mined, architecting activities supported, approaches and tools used, and challenges faced. Method: A Systematic Mapping Study (SMS) has been conducted on the literature published between January 2006 and November 2021. Results: Of the 79 primary studies finally selected, 8 categories of architectural information have been mined, among which architectural description is the most mined architectural information; 12 architecting activities can be supported by the mined architectural information, among which architecture understanding is the most supported activity; 81 approaches and 52 tools were proposed and employed in mining architectural information; and 4 types of challenges in mining architectural information were identified. Conclusions: This SMS provides researchers with promising future directions and help practitioners be aware of what approaches and tools can be used to mine what architectural information from what sources to support various architecting activities.Comment: 68 pages, 5 images, 15 tables, Manuscript submitted to a Journal (2022

On the Concerns of Developers When Using GitHub Copilot

With the recent advancement of Artificial Intelligence (AI) and the emergence of Large Language Models (LLMs), AI-based code generation tools have achieved significant progress and become a practical solution for software development. GitHub Copilot, referred to as AI pair programmer, utilizes machine learning models that are trained on a large corpus of code snippets to generate code suggestions or auto-complete code using natural language processing. Despite its popularity, there is little empirical evidence on the actual experiences of software developers who work with Copilot. To this end, we conducted an empirical study to understand the issues and challenges that developers face when using Copilot in practice, as well as their underlying causes and potential solutions. We collected data from 476 GitHub issues, 706 GitHub discussions, and 184 Stack Overflow posts, and identified the issues, causes that trigger the issues, and solutions that resolve the issues when using Copilot. Our results reveal that (1) Usage Issue and Compatibility Issue are the most common problems faced by Copilot users, (2) Copilot Internal Issue, Network Connection Issue, and Editor/IDE Compatibility Issue are identified as the most frequent causes, and (3) Bug Fixed by Copilot, Modify Configuration/Setting, and Use Suitable Version are the predominant solutions. Based on the results, we delve into the main challenges users encounter when implementing Copilot in practical development, the possible impact of Copilot on the coding process, aspects in which Copilot can be further enhanced, and potential new features desired by Copilot users

Technical Debt Management in OSS Projects: An Empirical Study on GitHub

Technical debt (TD) refers to delayed tasks and immature artifacts that may bring short-term benefits but incur extra costs of change during maintenance and evolution in the long term. TD has been extensively studied in the past decade, and numerous open source software (OSS) projects were used to explore specific aspects of TD and validate various approaches for TD management (TDM). However, there still lacks a comprehensive understanding on the practice of TDM in OSS development, which penetrates the OSS community's perception of the TD concept and how TD is managed in OSS development. To this end, we conducted an empirical study on the whole GitHub to explore the adoption and execution of TDM based on issues in OSS projects. We collected 35,278 issues labeled as TD (TD issues) distributed over 3,598 repositories in total from the issue tracking system of GitHub between 2009 and 2020. The findings are that: (1) the OSS community is embracing the TD concept; (2) the analysis of TD instances shows that TD may affect both internal and external quality of software systems; (3) only one TD issue was identified in 31.1% of the repositories and all TD issues were identified by only one developer in 69.0% of the repositories; (4) TDM was ignored in 27.3% of the repositories after TD issues were identified; and (5) among the repositories with TD labels, 32.9% have abandoned TDM while only 8.2% adopt TDM as a consistent practice. These findings provide valuable insights for practitioners in TDM and promising research directions for further investigation.Comment: 15 pages, 8 images, 10 tables, Manuscript submitted to a Journal (2022

Managing technical debt in software architecture

Architectural technical debt (ATD) is mainly incurred by architecture decisions that intentionally or unintentionally compromise system-wide quality attributes, particularly maintainability and evolvability. ATD needs to be systematically managed in an appropriate manner, in order to improve the health of the software architecture and optimize the cost of system maintenance and evolution in the long term. Currently, there is a lack of an overall process for systematically managing ATD, as well as approaches for concrete ATD management activities, particularly for ATD identification, measurement, and documentation. First, we conducted a systematic mapping study on technical debt (TD) and its management, to obtain a comprehensive understanding on the TD concept and current state of the research of TD management. Second, based on the mapping study results, we proposed a conceptual model of ATD, and developed an ATD management process integrating this model, in order to facilitate the architecture decision making and evaluation in a value-oriented perspective. Third, we proposed an ATD identification approach based on architecture decisions and change scenarios to overcome the shortcomings of existing source code analysis-based approaches. An industrial case study shows that the proposed approach is effective and easy to use. Fourth, we provided evidence that two modularity metrics, namely Index of Package Changing Impact and Index of Package Goal Focus, can be used to indicate the amount of ATD. Finally, we proposed six architecture viewpoints related to ATD, to document ATD. An industrial case study showed that the ATD viewpoints can effectively support ATD documentation